Repository navigation
fix(thrift): honor per-statement rowLimit - #623
vuanhphung wants to merge 2 commits into
Conversation
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Terra), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Medium
Looks good — 1 low concern. Thrift now forwards rowLimit as resultRowLimit, and its Number coercion matches the kernel path (KernelSessionBackend.ts:311). Unset requests stay uncapped, and only statementConf still triggers the warning.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Terra added findings it doesn't mention — see the findings below.
Other findings
- 🟡 Medium — The new tests inspect a stubbed request but the repository's e2e suite does not exercise
rowLimit, so it cannot catch regressions in actual capped results or fetch exhaustion. Add a warehouse e2e test for capped and uncapped results, including zero and a multi-fetch case. (Found by GPT-6.1 Terra · the judge couldn't confirm it)
|
Re: the Medium e2e-coverage finding: live warehouse coverage is in the shared conformance suite, not just the mocked unit tests: RESULTFETCH-020..023 at the tested coverage head. All four XPASS on final head f0fa993 after checking wire caps, exact ordered rows, inline/CloudFetch delivery, payload integrity, and fetch exhaustion (including cap 100003 over 200000). Main reproduces all four bugs. Eight additional live checks on this head confirm numeric-string and zero/null/NaN/fraction behavior across Thrift and SEA. I’m retaining the assignment’s shared-conformance coverage rather than duplicating it in this repository; the expected-failure wrappers will be flipped separately after merge. No uncovered runtime regression is demonstrated by this finding. This comment was generated with GitHub MCP. |
Thrift accepted
executeStatement(sql, { rowLimit })but dropped the server-side cap:rowLimit: 7overRANGE(20)returned all 20 rows. Forward explicitly set values as the existing ThriftresultRowLimiti64 field, keep unset requests uncapped, and warn only for unsupportedstatementConf. Coerce withNumberlike the kernel: numeric strings become numbers;null,NaN, and values truncated to zero produce an empty result. Document that upgrade effect, the supported Thrift range, invalid-value failures, and older servers ignoring the field. Fetch sizing and kernel behavior are unchanged.Verified lint, Prettier, type-check, build, and 1,808 unit tests, including numeric-string/null coercion regressions. The inherited
CODEX_CIenvironment exposes an existing agent-attribution test failure on both main and this branch; clearing only that variable makes the full suite pass. Live repository e2e: 11 passing, no skips. A fresh npm-tarball consumer type-checks and fetches the exact seven ordered rows; all four Thrift tripwires XPASS again on the final head, and eight live coercion checks pass across Thrift/SEA.Coverage source: databricks/databricks-driver-test#2161. At coverage head
152dbaf5, three main runs reproduce all four Thrift bugs; three fixed runs produce all 12 intended XPASS results after wire, exact-row, inline/CloudFetch, and exhaustion assertions. SEA passes all 12 cases. Wire cap / returned rows:7/7,100003/100003,11/11,10/7. Final-head driver-test passthrough CI run 44282933469 passes Thrift, SEA, and SEA-Reyden.Fresh-process execute+fetchAll medians, main → fix, three runs per cell with alternating build order on the shared warehouse: Thrift 100k rows 501 → 472 ms, CloudFetch 2392 → 2494 ms; SEA 402 → 405 ms, 2076 → 1775 ms. The coercion follow-up leaves this uncapped benchmark path unchanged; shared-warehouse variation is not an isolated throughput conclusion.
Security Scan's
picomatch@2.3.2finding (GHSA-fc5q-39xc-vjpf) also fails pristine main; the dependency lockfile is unchanged by this PR.Cross-driver state
setMaxRows(default0, unlimited);DatabricksThriftServiceClient.java:313,DatabricksSdkClient.java:961@d5bdaa3b.cursor(row_limit=None);thrift_backend.py:1162,sea/models/requests.py:48,kernel/client.py:668@b8259e4b.WithMaxRowsis page sizing, not a statement cap;connector.go:429,internal/backend/backend.go:120@d9008a9e.rowLimitdefaults unset/unlimited; explicit0returns no rows on both, unlike JDBC'ssetMaxRows(0).StatementSpec.row_limit: Option<i64>(defaultNone);src/statement/spec.rs:217,napi/src/connection.rs:217@ pinneddac32e2b.Adversarial review: round 1 converged with no high findings; all four remaining documentation/coercion findings are fixed.
This pull request and its description were written by Isaac.